• AirBnB is a website that allows hosts to rent their homes to travelers and tourists.
• AirBnB can maximize its business by ensuring it has enough listings in different neighborhoods.
• AirBnB has asked you, a data analytics consultant, to help them identify neighborhoods in Boston where they should focus on recruitment through advertising and incentives.
• They have given you a complex, multi-part data set to answer this question.

• Work with real and interesting data
• Create appealing and informative visualizations
• Draw useful conclusions

David Robinson

1. Look left.
2. Look right.
3. Pick a partner.
4. Give them a professional, yet friendly smile. You are going to need them soon.

• R without any packages offers ways to do most of the things we will see today.
• But base R is not a very good programming language.
• So, we will use….

• "Programs must be written for people to read and only incidentally for machines to execute" - Hal Abelson

The Tidyverse is a set of packages that promote code which is:

• Easy to read and write
• Highly performant
• Consistent across the data science workflow

When you master the Tidyverse, you spend more time thinking about your problem and less time thinking about your code.

AirBnB listings, schedule, and review text for the Boston area, for a time period…that you will check.

library(tidyverse)
listings <- read_csv('../data/listings.csv')
calendar <- read_csv('../data/calendar.csv')

• Question: What are the "nicest" places to stay in Jamaica Plain?
• Task: Construct a table of listings in JP, sorted in descending order by rating.

Conceptually, we need to: filter to only JP listings, arrange the listings by rating, and select only the columns we want to see. Let's write some code!

Working with your partner, please rewrite the JP code using the pipe operator. Here's the first line to get you started:

listings %>% 
    filter(neighbourhood == 'Jamaica Plain')
# ...

listings %>% 
    filter(neighbourhood == 'Jamaica Plain') %>%      # filter needs a logical test
    arrange(desc(review_scores_rating)) %>%           # desc() makes descending order 
    select(neighbourhood, name, review_scores_rating)

You are going to spend a long weekend in Back Bay with 50 of your closest friends.

Working with your partner, modify your code slightly to construct a table of the listings in Back Bay, sorted by the number of people who can stay there. You may need to use glimpse to see which columns you'll want to use.

listings %>% 
    filter(neighbourhood == 'Back Bay') %>% 
    arrange(accommodates) %>% 
    select(neighbourhood, name, accommodates, price)

Modify the summary table as follows:

1. Group by property_type as well as neighborhood. How does this impact the output?
2. Add a new column to the table with the average rate per person for WEEKLY rentals, using the weekly_price column.
3. Add a new column giving the total "capacity" for each neighborhood, given as the total number of beds in rentals in that neighborhood.

summary_table <- listings %>% 
    mutate(price = price %>% gsub('\\$|', '',.) %>% as.numeric(),
           price_per = price / accommodates,
           weekly_price = weekly_price %>% gsub('\\$|', '',.) %>% as.numeric(),
           weekly_price_per = weekly_price / accommodates)  %>% 
    group_by(neighbourhood, property_type) %>% 
    summarize(n = n(), 
              mean_rating = mean(review_scores_rating, na.rm = TRUE),
              price_per = mean(price_per, na.rm = TRUE),
              weekly_price_per = mean(weekly_price_per, na.rm = T),
              capacity = sum(beds)) 

ranked_summary_table <- summary_table %>% 
    mutate(rank = min_rank(desc(n))) %>% 
    filter(rank <= 3) %>% 
    arrange(neighbourhood, rank)

calendar %>% 
    summarise(earliest = min(date), 
           latest = max(date))

## # A tibble: 1 x 2
##     earliest     latest
##       <date>     <date>
## 1 2016-09-06 2017-09-05

Pretty current. But what if we want to focus on listings that have been active in the last three months?

Construct a table from the calendar data giving the listings that had a valid listed date between June 1st, 2016 and today. You determine what "valid" means in this context.

Hint: You can represent a date using the function

lubridate::mdy('6/1/2016')

## [1] "2016-06-01"

You can also use lubridate::today(). You can use max() to get the most recent date.

current_table <- calendar %>% 
    filter(!is.na(price), 
           date < lubridate::today(),
           date > lubridate::mdy('6/1/2016')) %>%
    group_by(listing_id) %>% 
    summarise(last_active = max(date))

The information we need is distributed between two tables – how can we get there?

We need a key that tells us which calendar rows correspond to which listings.

listings$id corresponds to calendar_listing$id

The join family of functions lets us add columns from one table to another using a key.

• x %>% left_join(y) : most common, keeps all rows of x but not necessarily y.
• x %>% right_join(y) : keeps all rows of y but not necessarily x.
  
• x %>% outer_join(y) : keeps all rows of both x and y
• x %>% full_join(y) : keeps only rows of x that match in y and vice versa.

We'll use left_join for this case.

Graphical excellence is the well-designed presentation of interesting data – a matter of substance, of statistics, and of design. Graphical excellence consists of complex ideas communicated with clarity, precision, and efficiency. – Edward Tufte

A grammar is a set of components (ingredients) that you can combine to create new things. Many grammars have required components: if you're missing one, you're doing it wrong. In baking….

• Puts the gg in ggplot2.
• Formulated by Leland Wilkinson.
• Implemented in code by Hadley Wickham, now part of the tidyverse

• Data: almost always a data_frame
• Aesthetic mapping: relation of data to chart components.
• Geometry: specific visualization type? E.g. line, bar, heatmap?
• Statistical transformation: how should the data be transformed or aggregated before visualizing?
• Theme: how should the non-data parts of the plot look?
• Misc. other options.

Data, aesthetics, and geometries are the required grammatical components that you always need to specify.

The following code computes the average price of all listings on each day in the data set:

average_price_table <- calendar %>% 
    mutate(price = price %>% gsub('\\$|', '',.) %>% as.numeric()) %>% 
    group_by(date) %>% 
    summarise(mean_price = mean(price, na.rm = TRUE))

Use geom_line() to visualize these prices with time on the x-axis and price on the y-axis.

 average_price_table %>% 
    ggplot() + 
    aes(x = date, y = mean_price) + 
    geom_line()

Using the summary_table object you created earlier, make a bar chart showing the number of apartments by neighbourhood. In this case, the correct geom to use is geom_bar(stat = 'identity').

We are going to make a simple business intelligence (BI) dashboard for AirBnB, using the wrangling and visualization skills we have developed in this session. You will use this dashboard to lead a meeting with decision-makers on where to prioritize host recruitment efforts.

1. You have added your names to the dashboard
2. You have replaced "Box 1", "Box 2", and "Box 3" with informative subtitles
3. You have created two original visualizations aimed at supporting a clear message about where AirBnB should recruit hosts.
4. You have written brief commentary clarifying your message.
5. You have submitted your dashboard to the shared Dropbox.

"In my experience, the vast majority of graphing agony is due to insufficient data wrangling." - Jenny Bryan

• Data Wrangling Cheatsheet